---
subtitle: >-
  Describes how to use the Opik LLM gateway and how to integrate with the Kong
  AI Gateway
---

An LLM gateway is a proxy server that forwards requests to an LLM API and returns the response. This is useful for when you want to centralize the access to LLM providers or when you want to be able to query multiple LLM providers from a single endpoint using a consistent request and response format.

The Opik platform includes a light-weight LLM gateway that can be used for **development and testing purposes**. If you are looking for an LLM gateway that is production ready, we recommend looking at the [Kong AI Gateway](https://docs.konghq.com/gateway/latest/ai-gateway/).

## The Opik LLM Gateway

The Opik LLM gateway is a light-weight proxy server that can be used to query different LLM API using the OpenAI format.

In order to use the Opik LLM gateway, you will first need to configure your LLM provider credentials in the Opik UI. Once this is done, you can use the Opik gateway to query your LLM provider:

<Tabs>
    <Tab value="Opik Cloud" title="Opik Cloud">
    ```bash
    curl -L 'https://www.comet.com/opik/api/v1/private/chat/completions' \
    -H 'Content-Type: application/json' \
    -H 'Accept: text/event-stream' \
    -H 'Comet-Workspace: <OPIK_WORKSPACE>' \
    -H 'authorization: <OPIK_API_KEY>' \
    -d '{
        "model": "<LLM_MODEL>",
        "messages": [
            {
                "role": "user",
                "content": "What is Opik ?"
            }
        ],
        "temperature": 1,
        "stream": false,
        "max_tokens": 10000
    }'
    ```
    </Tab>
        <Tab value="Opik self-hosted" title="Opik self-hosted">
    ```bash
    curl -L 'http://localhost:5173/api/v1/private/chat/completions' \
    -H 'Content-Type: application/json' \
    -H 'Accept: text/event-stream' \
    -d '{
        "model": "<LLM_MODEL>",
        "messages": [
            {
                "role": "user",
                "content": "What is Opik ?"
            }
        ],
        "temperature": 1,
        "stream": false,
        "max_tokens": 10000
    }'
    ```
    </Tab>
</Tabs>

<Warning>
  The Opik LLM gateway is currently in beta and is subject to change. We recommend using the Kong AI gateway for
  production applications.
</Warning>

## Kong AI Gateway

[Kong](https://docs.konghq.com/gateway/latest/) is a popular Open-Source API gatewy that has recently released an AI Gateway.
If you are looking for an LLM gateway that is production ready and supports many of the expected enterprise use cases
(authentication mechanisms, load balancing, caching, etc), this is the gateway we recommend.

You can learn more about the Kong AI Gateway [here](https://docs.konghq.com/gateway/latest/ai-gateway/).

We have developed a Kong plugin that allows you to log all the LLM calls from your Kong server to the Opik platform.
The plugin is available for enterprise customers. Please contact our support team for access.

Once the plugin is installed, you can enable it by running:

```bash
curl -is -X POST http://localhost:8001/services/{serviceName|Id}/plugins \
    --header "accept: application/json" \
  --header "Content-Type: application/json" \
  --data '
  {
    "name": "opik-log",
    "config": {
      "opik_api_key": "<Replace with your Opik API key>",
      "opik_workspace": "<Replace with your Opik workspace>"
    }
  }'
```

For more information about the Opik Kong plugin, please contact our support team.

Once configured, you will be able to view all your LLM calls in the Opik dashboard:

<Frame>
  <img src="/img/production/opik-kong-gateway.png" />
</Frame>
